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Abstract 


In  the  analog  VLSI  implementation  of  neural  systems,  it  is  sometimes  convenient  to  build 
lateral  inhibition  networks  by  using  a  locally  connected  on-chip  resistive  grid  to 
interconnect  active  elements.  A  serious  problem  of  unwanted  spontaneous  oscillation 
often  arises  with  these  circuits  and  renders  them  unusable  in  practice.  This  paper  reports 
on  criteria  that  guarantee  these  and  certain  other  systems  will  be  stable,  even  ^ough  the 
values  of  designed  elements  in  the  resistive  grid  may  be  imprecise  and  ^e  location  and 
values  of  parasitic  elements  may  be  unknown.  The  method  is  based  on  a  rigorous, 
somewhat  novel  mathematical  analysis  using  Tellegen’s  theorem  from  electrical  circuits 
and  the  idea  of  a  Popov  multiplier  from  control  theory.  The  criteria  are  local  in  that  no 
overall  analysis  of  the  interconnected  system  is  required  for  their  use,  empirical  in  that  they 
involve  only  measurable  frequency  response  data  on  the  individual  cells,  and  robust  in  that 
they  are  insensitive  to  network  topology  and  to  unmodelled  parasitic  resistances  and 
capacitances  in  the  interconnect  network-  Certain  results  are  robust  in  the  additional  sense 
that  specified  nonlinear  elements  in  the  grid  do  not  affect  the  stability  criteria.  The  results 
are  designed  to  be  applicable,  with  further  development,  to  complex  and  incompletely 
modelled  living  neiuaj  systems. 


DTIC 

ELECTE 


FEB  061988r  j 

y 


H 


Microsystems 
Research  Center 


Rcc'^  90.321 


Massachusetts 
Institute 
of  Technology 


Cambridge 

Massachusetts 

02139 


Telephone 
(617)  253-8138 


8  9  2  L>  OK 


Acknowledgements 

To  appear  in  Neural  Computation,  Vol,  1,  No.  1,  Spring  1989.  This  research  was  supported 
in  part  by  the  Defense  Advanced  Research  Projects  Agency  under  contract  number 
N(X)014-87-K-0825  and  the  National  Sdeuce  Foundation  under  contract  MlP-8814612. 


Author  Information 

Wyatt:  Electrical  Engineering  and  Computer  Science,  MTT,  Room  36-864,  Cambridge, 
MA  02139.  (617)  253-6718. 

Standley:  Electrical  Engineering  and  Computer  Science,  MIT,  Room  36-863,  Cambridge, 
MA  02139.  (617)  253-2631. 


Copyright®  1988  MIT.  Memos  in  this  series  are  for  use  inside  MIT  and  are  not 
considered  to  be  published  merely  by  virtue  of  appearing  in  this  series.  This  copy  is  for 
private  circulation  only  and  may  not  be  further  copied  or  distributed,  except  for 
government  purposes,  if  the  paper  acknowledges  U.  S.  Government  sponsorship. 
References  to  this  work  should  be  either  to  the  published  version,  if  any,  or  in  the  form 
"private  communication.”  For  information  about  the  ideas  expressed  herein,  contact  the 
author  directly.  For  information  about  this  series,  contact  Microsystems  Research 
Center,  Room  39-321,  MIT,  Cambridge,  MA  02139;  (617)  253-8138. 


To  appear  in  Neural  Computation, 
vol.  1,  no.  1,  Spring  1989 


Criteria  for  Robust  Stability  In  A  Class  Of  Lateral  Inhibition 
Networks  Coupled  Through  Resistive  Grids 


John  L.  Wyatt,  Jr.  zuid  David  L.  Standley 
Department  of  Electrical  Engineering  and  Computer  Science 
Massachusetts  Institute  of  Technology 
Cambridge,  MA  02139 


Abstract 

In  the  analog  VLSI  implementation  of  neural  systems,  it  is  sometimes 
convenient  to  build  lateral  inhibition  networks  by  using  a  locally  connected 
on-chip  resistive  grid  to  interconnect  active  elements.  A  serious  problem 
of  unwanted  spontaneous  oscillation  often  arises  with  these  circuits  smd 
renders  them  unusable  in  practice.  This  paper  reports  on  criteria  that 
guarantee  these  and  certain  other  systems  will  be  stable,  even  though  the 
values  of  designed  elements  in  the  resistive  grid  may  be  imprecise  and  the 
location  zmd  values  of  parasitic  elements  may  be  unknown.  The  method  is 
based  on  a  rigorous,  somewhat  novel  mathematical  aneJysis  using  Tellegen’s 
theorem  [1]  from  electriczd  circxiits  and  the  idea  of  a  Popov  multiplier  [2,3] 
from  control  theory.  The  criteria  are  local  in  that  no  overall  analysis  of 
the  interconnected  system  is  reqtiired  for  their  use,  empirical  in  that  they 
involve  only  measiuable  frequency  response  data  on  the  individual  cells,  and 
robust  in  that  they  are  insensitive  to  network  topology  and  to  unmodelled 
parasitic  resistances  and  capacitances  in  the  interconnect  network.  Certain 
results  are  robust  in  the  additional  sense  that  specified  nonlinear  elements 
in  the  grid  do  not  affect  the  stability  criteria.  The  results  are  designed 
to  be  applicable,  with  further  development,  to  complex  and  incompletely 
modelled  living  neural  systems. 
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I.  Introduction 

In  the  VLSI  implementation  of  neural  systems,  it  is  convenient  to  build 
lateral  inhibition  and  certain  other  types  of  networks  by  locally  intercon¬ 
necting  active  cells  through  an  on-chip  resistive  grid.  Linear  resistors  fab¬ 
ricated  in,  e.g.,  polysilicon,  coixld  yield  a  very  compact  realization,  and 
nonlineM  resistive  grids,  made  from  MOS  transistors,  have  been  foimd  use¬ 
ful  for  image  segmentation  [4].  Networks  of  this  type  can  be  divided  into 
two  clztsses:  feedback  systems  and  feedforward-only  systems.  In  the  feed¬ 
forward  case  one  set  of  amplifiers  imposes  signal  voltages  or  currents  on 
the  grid  and  another  set  reads  out  the  resulting  response  for  subsequent 
processing,  while  the  same  amplifiers  both  “write  to”  the  grid  and  “read 
from”  it  in  a  feedback  arrangement.  Feedforward  networks  of  this  type  are 
inherently  stable,  but  feedback  networks  need  not  be. 

A  practiced  example  is  one  of  Mahowald  eind  Mead’s  retina  chips  [5,6] 
that  achieve  edge  enhancement  by  means  of  lateral  inhibition  through  a 
resistive  grid.  Figure  la  shows  a  single  cell  in  an  earlier  version  of  this  chip, 
and  Fig.  lb  illustrates  the  network  of  interconnected  cells.  Experiment  has 
shown  that  the  individual  cells  in  this  system  are  open-circuit  stable  and 
remain  stable  when  the  output  of  amplifier  #  2  is  connected  to  a  voltage 
source  through  a  resistor,  but  the  interconnected  system  oscillates  so  badly 
that  the  earlier  design  is  scarcely  usable^  [7].  Such  oscillations  can  readily 
occur  in  most  resistive  grid  circuits  with  active  elements  and  feedback, 
even  when  each  individual  cell  is  quite  stable.  Analysis  of  the  conditions  of 
instability  by  conventional  methods  appears  hopeless,  since  the  number  of 
simultaneously  active  feedback  loops  is  enormous. 

This  paper  reports  a  practical  design  approach  that  rigorously  gxmran- 
tees  such  a  system  will  be  stable  if  the  active  cells  meet  certain  criteria. 
The  work  begins  with  the  naive  observation  that  the  system  would  be  stable 
if  we  could  design  each  individual  cell  so  that,  although  internedly  active, 
it  acts  like  a  passive  system  as  seen  from  the  resistive  grid.  The  design 
goal  in  that  case  would  be  that  each  cell’s  output  impedance  should  be  a 
positive-real  [2,3,8]  function.  This  is  sometimes  possible  in  practice;  we  will 

^  The  later  design  reported  in  [5]  avoids  stability  problems  altogether,  at  a  small  cost 
in  performance,  by  redesigning  the  circuits  to  passively  sense  the  grid  voltage  in  a  “feed- 
torward”  style  as  described  above. 
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show  that  the  original  network  in  Fig.  la  would  satisfy  this  condition  in 
the  absence  of  certain  parasitic  elements.  Furthermore,  it  is  a  condition 
one  can  verify  experimentally  by  frequency-response  measxirements. 

It  is  obvious  that  a  collection  of  cells  that  appear  passive  at  their  ter¬ 
minals  will  form  a  stable  system  when  interconnected  through  a  passive 
medixim  such  as  a  resistive  grid,  and  that  the  stability  of  such  a  system  is 
robust  to  perturbations  by  passive  parasitic  elements  in  the  network.  The 
work  reported  here  goes  beyond  that  observation  to  provide  i)  a  demonstra¬ 
tion  that  the  passivity  or  positive-real  condition  is  much  stronger  than  we 
actually  need  and  that  weaker  conditions,  more  easily  achieved  in  practice, 
suffice  to  guarantee  robust  stability  of  the  linear  active  network  model,  and 
ii)  an  extension  of  the  analysis  to  the  nonlinear  domain  that  furthermore 
rules  out  sustained  large-signal  oscillations  under  certain  conditions.  A  key 
feature  of  the  integrated  circuit  environment  that  makes  these  results  ap¬ 
plicable  is  the  almost  total  absence  of  on-chip  inductance.  While  the  cells 
can  appear  inductive,  as  in  Fig.  3c,  the  absence  of  inductance  in  our  grid 
models  makes  these  theorems  possible. 

Note  that  these  results  do  not  apply  directly  to  networks  created  by 
interconnecting  neuron-like  elements,  as  conventionally  described  in  the 
literatvure  on  artificial  nexiral  systems.  The  “neurons”  in,  e.g.,  a  Hopfield 
network  [9]  are  unilateral  2-port  elements  in  which  the  input  and  output 
are  both  voltage  signals.  The  input  voltage  uniquely  and  instantemeously 
determines  the  output  voltage  of  such  a  nemon  model,  but  the  output 
can  only  affect  the  input  via  the  resistive  grid.  In  contrast,  the  cells  in 
our  system  are  l-port  electrical  elements  (temporarily  ignoring  the  optical 
input  channel)  in  which  the  port  voltage  and  port  current  axe  the  two 
relevant  signals,  and  each  signal  affects  the  other  through  the  cell’s  internal 
dynamics  (modelled  as  a  Thevenin  equivalent  impedance)  as  well  as  through 
the  grid’s  response. 

It  is  apparent  that  uncontrolled  spontaneous  oscillation  is  a  potential 
problem  in  living  neural  systems,  which  typically  also  consist  of  active  ele¬ 
ments  arranged  in  feedback  loops.  Biological  systems  have  surely  solved  the 
same  problem  we  attack  in  this  paper.  It  is  reeisonable  to  believe  that  sta¬ 
bility  has  strongly  constrained  the  set  of  network  configurations  nature  has 
produced.  Whatever  Natures’s  solutions  may  be,  we  suspect  they  have  at 
least  three  features  in  common  with  the  ones  proposed  here:  1)  robustness 
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in  the  face  of  wide  component  variation  and  the  presence  of  parasitic  net¬ 
work  elements,  2)  reliance  on  empirical  data  rather  than  anything  we  would 
recognize  os  a  theory  or  analytic  method,  3)  stability  strategies  based  on 
predominantly  local  information  available  to  each  network  element. 

Several  reports  on  this  work  have  appeared  and  will  appear  in  [10-14] 
during  its  development;  a  longer  tutorial  exposition  will  be  given  in  the 
second  printing  of  [6]. 

II.  The  Linear  Theory 


Terminology 

The  output  impedance  of  a  linear  system  is  a  measure  of  the  voltage 
response  due  to  a  change  in  output  current  while  the  input  (light  intensity 
in  this  case)  is  held  constant.  This  standard  electrical  engineering  concept 
will  play  a  key  role  here.  Figure  2a  illustrates  one  experimental  method 
for  measuring  the  output  impedance,  and  Fig.  2b  is  a  standard  graphical 
representation  of  an  imjjedance,  known  as  a  Nyquist  diagram.  Similar  plots 
have  been  used  in  experimental  physiology  [15] . 

In  the  context  of  this  work,  an  impedance  is  said  to  be  positive-real 
[2,3,8]  if  it  is  stable  (i.e.,  has  no  poles  or  zeroes  in  the  right-half  plame)  and 
its  Nyquist  diagram  lies  entirely  in  the  right-half  plane  (i.e.,  in  the  language 
of  complex  numbers,  Re{Z{iui)}  >  0  for  all  purely  sinusoideil  frequencies 
(jj).  Figure  2a  is  an  example,  while  the  system  represented  in  Fig.  4  is 
stable  but  not  positive-real. 

A  deep  link  between  positive-real  functions,  physical  networks  and  pas¬ 
sivity  is  established  by  the  classical  result  in  linear  circuit  theory  which 
states  that  H{3)  is  positive-real  if  and  only  if  it  is  possible  to  synthesize 
a  2-terminal  network  of  positive  linear  resistors,  capacitors,  inductors  and 
ideal  transformers  that  has  H{s)  as  its  driving-point  impedance  [8]. 

This  work  was  originally  motivated  by  the  following  linear  analysis  of  a 
model  for  the  circuit  in  Fig.  la.  For  an  initial  approximation  to  the  output 
impedance  of  the  cell  we  use  the  elementary  model  shown  in  Fig.  3a  for  the 
amplifiers  and  simplify  the  circxiit  topology  within  a  single  cell  as  shown  in 
Fig.  3b. 

Straightforward  calculations  show  that  the  output  impedance  is  given 
by 
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=  +  +  +  (1) 

This  is  a  positive-real  impedance  that  could  be  realized  by  a  passive 
network  of  the  form  shown  in  Fig.  3c,  where 

•^1  ~  (^mj  "t"  -^oj  )  1-^2  ~  )  ,  and  Zi  =  Cq^/ ffrnidmi  •  (2) 

Of  course  this  model  is  oversimplified,  since  the  circuit  does  oscillate. 
Transistor  parasitics  and  layout  parasitics  cause  the  output  impedance  of 
the  individual  active  cells  to  deviate  from  the  form  given  in  eqs.  (1)  and 
(2),  and  any  very  accurate  model  will  necessarily  be  quite  high  order.  The 
following  theorem  shows  how  far  one  can  relax  the  positive-real  condition 
and  still  guarantee  that  the  entire  network  is  robustly  stable.  It  obviously 
appUes  to  a  much  wider  range  of  linear  networks  than  has  been  discussed 
here. 

A  linear  network  is  said  to  be  stable  if  for  any  initiaJ  condition  the 
transient  response  converges  asymptotically  to  a  constant. 

Theorem  1 

Consider  the  class  of  linear  networks  of  arbitrary  topology,  consisting  of  any 
number  of  positive  2-terminal  resistors  and  capacitors  and  of  N  lumped 
linear  impedances  Zn{s),n  =  1,2,  ...,JV,  that  are  open-  and  short-circuit 
stable  in  isolation,  i.e.,  that  have  no  poles  or  zeroes  in  the  closed  right-half 
plane.  Every  such  network  is  stable  if  at  each  frequency  w  >  0  there  exists 
a  phzise  angle  ^(w)  such  that  0  >  9{u>)  >  —90®  and  —  0(iu;)|  < 

90®,  n  =  l,2,...,iV. 

An  equivalent  statement  of  this  last  condition  is  that  the  Nyquist  plot 
of  each  cell’s  output  impedance  for  u;  >  0  never  intersects  the  2nd  quadrant 
of  the  complex  plane  (Fig.  4  is  an  example),  and  that  no  two  cells’  output 
impedance  phase  angles  can  ever  differ  by  as  much  as  180®.  If  all  the  active 
cells  are  designed  identically  and  fabricated  on  the  same  chip,  their  phase 
angles  should  track  fairly  closely  in  practice,  and  thus  this  second  condition 
is  a  natural  one. 
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The  theorem  is  intuitively  reasonable  and  serves  as  a  practicsJ  design 
goal.  The  assumptions  guarantee  that  the  cells  caimot  resonate  with  one 
another  at  any  purely  sinusoidal  frequency  s  =  ju  since  their  phase  angles 
can  never  differ  by  as  much  as  180“,  and  they  can  never  resonate  with  the 
resistors  and  capacitors  since  they  can  never  appear  simultaneously  aictive 
and  inductive  at  any  sinusoidal  frequency.  A  more  advanced  argument  [14] 
shows  that  exponentially  growing  instabilities  zdso  ruled  out. 


6 


III.  stability  Result  for  Networks  with  Nonlinear  Resistors  and 

Capacitors 

The  previous  results  for  linear  networks  can  afford  some  limited  insight 
into  the  behavior  of  nonlinear  networks.  If  a  linearized  model  is  stable, 
then  the  equilibrium  point  of  the  original  nonlinear  network  must  be  locally 
stable.  But  the  result  in  this  section,  in  contrast,  applies  to  the  full  nonlinear 
circuit  model  and  allows  one  to  conclude  that  in  certain  circumstances  the 
network  cannot  oscillate  or  otherwise  fail  to  converge  even  if  the  initial 
state  is  arbitrarily  far  from  the  equilibrium  point.  Figure  4  introduces  the 
Popov  criterion,  which  is  the  basis  of  the  following  theorem.  This  is  the 
first  nonlinear  result  of  its  type  that  requires  no  assumptions  on  the  network 
topology. 

Theorem  2 

Consider  any  network  consisting  of  nonlinear  resistors  and  capacitors  and 
linear  active  cells  with  output  impedances  Zn(5),n  =  1,2,  Suppose 

a)  the  nonlinear  resistor  and  capacitor  characteristics,  ij  =  gj(vj) 
and  qit  =  hk{vic),  respectively,  are  monotone  increasing  continuously 
differentiable  functions,  and 

b)  the  impedances  Zn{s)  all  satisfy  the  Popov  criterion  for  some  com¬ 
mon  value  of  r  >  0. 

Then  the  network  is  stable  in  the  sense  that,  for  any  initial  condition 
at  #  =  0, 


^  ]rf<<oo 

all  resistors  (^) 

and  capacitors 
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FIGURE  CAPTIONS 


Figure  1.  a)  This  photoreceptor  jmd  signal  processor  circuit,  using 
two  MOS  amplifiers,  realizes  spatial  lateral  inhibition  and  temporal 
sharpening  by  communicating  with  similar  cells  through  a  resistive 
grid.  The  resistors  will  often  be  nonlinear  by  design,  b)  Intercon¬ 
nection  of  cells  through  a  hexagonal  resistive  grid.  Cells  axe  drawn 
as  2-terminal  elements  with  the  power  supply  and  signal  output  lines 
suppressed.  The  voltage  on  the  capacitor  in  any  given  cell  is  affected 
both  by  the  local  light  intensity  incident  on  that  cell  and  by  the  capac¬ 
itor  voltages  on  neighboring  cells  of  identical  design.  The  necessary 
ingredients  for  instability  —  active  elements  and  signal  feedback  — 
are  both  present  in  this  system,  c)  Grid  resistors  with  a  nonlinear 
characteristic  of  the  form  i  =  tanh{v)  can  be  useful  in  image  segmen¬ 
tation  [4]. 

Figure  2.  a)  Simplified  experimental  measurement  of  the  output 
impedance  of  a  cell.  A  sinusoidal  current  i  =  Acos{(jjt)  is  injected  into 
the  output  and  the  voltage  response  v  =  Bcos(ujt  +  <i>)  is  measured. 
The  impedance,  which  has  magnitude  B/A  and  phase  <f>,  is  typically 
treated  as  a  complex  number  Z(iu;)  that  depends  on  the  firequency 
u>.  b)  Example  of  the  Nyquist  diagram  of  an  impedamce.  This  is  a 
plot  in  the  complex  plane  of  the  value  of  the  impedance,  meeisured  or 
calculated  at  purely  sinusoidal  frequencies,  ranging  from  zero  upward 
toward  infinity.  It  is  not  essential  to  think  of  Nyquist  diagrams  as 
representing  complex  numbers:  they  are  simply  polar  plots  in  which 
radius  represents  impedance  magnitude  and  angle  to  the  horizontal 
axis  represents  phase.  The  diagram  shown  here  is  the  Nyquist  plot  of 
the  positive-real  impedance  in  eq.  (1). 

Figure  3.  a)  Elementary  model  for  an  MOS  amplifier.  These  ampli¬ 
fiers  have  a  relatively  high  output  resistance,  which  is  determined  by  a 
bias  setting  (not  shown),  b)  Linearity  allows  this  simplification  of  the 
network  topology  for  the  circuit  in  Fig.  la  without  loss  of  information 
relevant  to  stability.  The  capacitor  in  Fig.  la  has  been  absorbed  into 
the  output  capax:itance  of  amp  #  2.  c)  Passive  network  realization  of 
the  output  impedance  given  in  eq.  (1)  for  the  network  in  b). 
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Figure  4.  Nyquist  diagram  of  an  impedance  that  satisfies  the  Popov 
criterion,  defined  as  follows.  A  linear  impedance  Z(s)  satisfies  the 
Popov  criterion  if  (1  +  rs)Z(s)  is  positive-real  for  some  r  >  0.  The 
“Popov  multiplier”  (1  -|-  rs)  modifies  the  Nyquist  diagram  by  stretch¬ 
ing  and  rotating  it  cotmterclockwise  for  a;  >  0.  The  impedance  plotted 
here  is  active  amd  thus  is  not  positive-real,  but  the  rotation  due  to  the 
(1  -f-  ra)  term  can  make  it  positive-real  for  an  appropriate  value  of 
r.  The  Popov  criterion  is  a  condition  on  the  linear  elements  that 
is  weaiker  than  passivity:  active  elements  satisfying  this  criterion  are 
shown  to  pose  no  danger  of  instability  even  when  nonlinear  resistors 
and  capacitors  are  present  in  the  grid. 
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